EM Algorithms for Probabilistic Mapping Networks

The Expectation-Maximization (EM) algorithm is a general technique for maximum likelihood estimation (MLE). In this paper we present several of the important theoretical and practical issues associated with Gaussian mixture mod-eling (GMM) within the EM framework. First, we propose an EM algorithm for estimating the parameters of a special GMM structure, named a probablistic mapping network (PMN), where the Gaussian probability density function is realized as an internal node. In this way, the EM algorithm is extended to deal with the supervised learning of a multicategory classiication problem and serve as a parameter estimator of the neural network Gaussian classiier. Then, a generalized EM (GEM) algorithm is developed as an alternative to the MLE problem of PMN. This is followed by a discussion on the computational considerations and algorithmic comparisons. It is shown that GEM converges faster than EM to the same solution space. The computational eeciency and the numerical stability of the training algorithm beneet from the well-established EM framework. The eeectiveness of the proposed PMN architecture and developed EM algorithms are assessed by conducting a set of speaker recognition experiments. INRIA Apprentissage de RRseaux Probabilistes par Algorithmes EM RRsumm : L'algorithme "Expectation-Maximisation" (EM) est une technique gg-nnrale pour l'estimation par maximum de vraisemblance. Ce document prrsente quelques aspects thhoriques et pratiques importants liis la moddlisation par mm-langes de gaussienne (GMM) dans le cadre EM. Nous proposons d'abord un algo-rithme EM pour estimer le parammtres d'un moddle GMM particulier, appell PMN ("probabilistic mapping network"), dans lequel la fonction de densitt de probabilitt gaussienne est calculle par un noeud interne du rrseau. Nons prrsentons ensuite un algorithm EM ggnnraliss (GEM) comme une solution alternative l'estimation par maximum de vraisemblance d'un PMN. Cette prrsentation est complltte par une discussion sur les aspects calculatoires comparrs des deux algorithmes EM et GEM. Nous montrons en particulier que GEM converge plus vite que EM vers le mmme espace de solutions. L'eecacitt pratique de l'architecture PMN et des algo-rithmes EM proposss est valuue travers un ensemble d'exemples pratiques dans le domaine de la reconnaissance de locuteurs.